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A two-person zero-sum differential game with unbounded controls is considered. Under proper coer- 
civity conditions, the upper and lower value functions are characterized as the unique viscosity solutions 
to the corresponding upper and lower Hamilton-Jacobi-Isaacs equations, respectively. Consequently, 
when the Isaacs' condition is satisfied, the upper and lower value functions coincide, leading to the ex- 
(-H , istence of the value function of the differential game. Due to the unboundedness of the controls, the 

corresponding upper and lower Hamiltonians grow super linearly in the gradient of the upper and lower 
value functions, respectively. A uniqueness theorem of viscosity solution to Hamilton-Jacobi equations 
involving such kind of Hamiltonian is proved, without relying on the convexity/concavity of the Hamil- 
tonian. Also, it is shown that the assumed coercivity conditions guaranteeing the finiteness of the upper 
04 ■ and lower value functions are sharp in some sense. 
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1 Introduction 



S^ ■ Let us begin with the following control system: 



I y{s) = f{s,y{s),Ui{s),U2{s)), se[t,T], 

I y{t) = ^- 

where / : [0, T] x R" x Ui x U2 —> R" is a given map. In the above, y{-) is the state trajectory taking values 
in R", and (mi(-), U2(-)) is the control pair taken from the set Ui^[t,T] x Z^^^[i,T] of admissible controls, 
defined by the following: 



Up[t,T] = {u, : [t,T] ^ U, I \\u,i-)\\L^^t,T) = [ f \u^is)rds 



T 

< 00 !•, i — 1,2, 

with Ui being a closed subset of R™' and with some (Ji > 1. We point out that Ui and U2 are allowed 
to be unbounded, and they could even be R"*^ and R™^, respectively. Hereafter, we suppress R™' in 
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||wi(')lli''>(t,T;R'"i) for notational simplicity and this will not cause confusion. The performance functional 
associated with (1.1) is the following: 

J(t,x;ui(-),M2(-)) = / 9{s,y{s),ui{s),U2{s))ds + h{y(T)), (1.2) 

with g : [0, T] x R" x f/i x C/2 ^ IR and h : R" ^- R being some given maps. 

The above setting can be used to describe a two-person zero-sum differential game: Player 1 wants to 
select a control wi(-) G W^^[i,T] so that the functional (1.2) is minimized and Player 2 wants to select a 
control U2{-) G U2^[t,T] so that the functional (1.2) is maximized. Therefore, J{t,x;ui{-),U2{-)) is a cost 
functional for Player 1 and a payoff functional for Player 2, respectively. If U2 is a singleton, the above is 
reduced to a standard optimal control problem. 

Under some mild conditions, for any initial pair {t,x) G [0,r] x R" and control pair (ui{-) , U2{-)) G 
Ui^[t,T] X U2^[t,T], the state equation (1.1) admits a unique solution j/(-) = y{- ;t,x,ui{-),U2{-)), and the 
performance functional J(i, x; ui{-), U2{-)) is well-defined. By adopting the notion of Elliott-Kalton strategies 
([11]), we can define the upper and lower value functions V^ : [0,T] x R" -^ R (see Section 3 for details). 
Further, when V {■ , •) are differentiable, they should satisfy the following upper and lower Hamilton- Jacobi- 
Isaacs (HJI, for short) equations, respectively: 



F±(i,x)+if±(i,x,T/±(t,x))=0, 
T/±(T,x) = /i(x), a;GR", 



{t,x) G [0,r] xR" 



(1.3) 



where H {t,x,p) are the so-called upper and lower Hamiltonians defined by the following, respectively: 



H+{t,x,p)= inf sup {p,f{t,x,ui,U2))+g{t,x,ui,U2) 

H^{t,x,p) ^ sup inf {p,f{t,x,ui,U2))+g{t,x,ui,U2) 
u2eU2 «iet/i L 



(t,x,p)G [0,r] xR"xR". (1.4) 



When the sets Ui and U2 are bounded, the above differential game is well-understood ([12, 16]): Under 
reasonable conditions, the upper and lower value functions V {■ , •) are the unique viscosity solutions to the 
corresponding upper and lower HJI equations, respectively. Consequently, in the case that the following 
Isaacs condition: 

H+{t,x,p)^H-{t,x,p), V(t,x,p) G [0,T] X R" X R", (1.5) 

holds, the upper and lower value functions coincide and the two-person zero-sum differential game admits 
the value function 

V{t,x) = V+{t,x) = V-{t,x), (t,a;) G [0,r] xR". (1.6) 

For comparison purposes, let us now take a closer look at the properties that the upper and lower value 
functions V {■ , •) and the upper and lower Hamiltonians H {■ , • , •) have, under classical assumptions. To 
this end, let us recall the following classical assumption: 

(B) Functions / : [0,T] x R" x C/i x C/2 ^ R", 5 : [0,T] x R" x J/i x C/2 -> R, and /i : R" -> R are 
continuous. There exists a constant L > and a continuous function lo : [0, 00) x [0, 00) — > [0, 00), increasing 
in each of its arguments and uj{r,0) — for all r > 0, such that for aU t,s £ [0,T], x,y G R", (mi,M2) G 

Ul X C/2, 

\f{t,x,ui,U2) - f{s,y,ui,U2)\ < L\x-y\ +uj{\x\ V \y\,\t- s\), 
\g{t,x,ui,U2) - g{s,y,ui,U2)\ < uj{\x\ V \y\, \x - y\ + \t - s\), 
\h{x)-hiy)\<uj{\x\V\y\,\x-y\), 
I \f{t,0,ui,U2)\ + \g{t,0,ui,U2)\ + \h{0)\ < L, 
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(1.7) 



where \x\ V \y\ = niax{|x|, |y|}. 

Condition (1.7) implies that the continuity and the growth of {t,x) n- {f{t,x,ui,U2), g(t,x,ui,U2)) are 
uniform in (ui, U2) G Ui xU2- This essentially will be the case if Ui and U2 are bounded (or compact metric 
spaces). Let us state the following proposition. 

Proposition 1.1. Under assumption (B), one has the following: 

(i) The upper and lower value functions V^{- , •) are well-defined continuous functions. Moreover, they 
arc the unique viscosity solutions to the upper and lower HJI equations (1.3), respectively. In particular, if 
Isaacs' condition (1.5) holds, the upper and lower value functions coincide. 

(ii) The upper and lower Hamiltonians H^{- , • , •) satisfy the following: For all t G [0, T], x, y,p,q G R", 

f \H^{t,x,p)-H^{t,y,q)\<L{l + \x\)\p-q\+uj{\x\y\y\,\x~y\), ^^ ^^ 

\ \H^{t,x,p)\<L{l + \x\)\p\+L + uji\xl\x\). 

Condition (1.8) plays an important role in the proof of the uniqueness of viscosity solution to HJI 
equations (1.3) ([5, 15]). Note that, in particular, (1.8) implies that p 1— > H^{t,x,p) is at most of linear 
growth. 

Unfortunately, the above property (1.8) fails, in general, when the control domains Ui and/or U2 is 
unbounded. To make this more convincing, let us look at a one-dimensional linear-quadratic (LQ, for short) 
optimal control problem (which amounts to saying that Ui — H and U2 ~ {0}). Consider the state equation 

y{s) =y(s) +u{s), s G [t,T], 

with a quadratic cost functional 

1. rT 



J(i,x;u(.)) = - / {\y{s)\' + \u{s)\^)ds+\y{T)\' 



Then the Hamiltonian is 



H{t, x,p) = inf p{x + u) + 
«eR L 



x^ p^ 

xn -\ . 

^2 2 



Thus, p H- > H{t,x,p) is of quadratic growth and (1.8) fails. 

Optimal control problems with unbounded control domains were studied in [2, 8]. Uniqueness of viscosity 
solution to the corresponding Hamilton- Jacobi-Bellman equation was proved by some arguments relying on 
the convexity/concavity of the corresponding Hamiltonian with respect to p. Recently, the above results were 
substantially extended to stochastic optimal control problems ([10]). On the other hand, as an extension of 
[24], two-person zero-sum differential games with (only) one player having unbounded control were studied 
in [20]. Some nonlinear Hoo problems can also be treated as such kind of differential games [19, 21]. Further, 
stochastic two-person zero-sum differential games were studied in [9] with one player having unbounded 
control and with the two players' controls being separated both in the state equation and the performance 
functional. 

The main purpose of this paper is to study two-person zero-sum differential games with both players hav- 
ing unbounded controls, and the controls of two players are not necessarily separated. One motivation comes 
from the problem of what we call the afhne-quadratic (AQ, for short) two-person zero-sum differential games, 
by which we mean that the right hand side of the state equation is affinc in the controls, and the integrand 
of the performance functional is quadratic in the controls (sec Section 2). This is a natural generalization 
of the classical LQ problems. For general two-person zero-sum differential games with (both players having) 
unbounded controls, under some mild coercivity conditions, the upper and lower Hamiltonians H {t,x,p) 
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are proved to be well-defined, continuous, and locally Lipschitz in p. Therefore, the upper and lower HJI 
equations can be formulated. Then we will establish the uniqueness of viscosity solutions to a general first 
order Hamilton-Jacobi equation which includes our upper and lower HJI equations of the differential game. 
Comparing with a relevant result found in [6] , the conditions we assumed here are a little different from theirs 
and we present a detailed proof for reader's convenience. By assuming a little stronger coercivity conditions, 
together with some additional conditions (guaranteeing the well-posedness of the state equation, etc.), we 
show that the upper and lower value functions can be well-defined and are continuous. Combining the above 
results, one obtains a characterization of the upper and lower value functions of the differential game as 
the unique viscosity solutions to the corresponding upper and lower HJI equations. Then if in addition, the 
Isaacs' condition holds, the upper and lower value functions coincide which yields the existence of the value 
function of the differential game. 

We would like to mention here that due to the unboundedncss of the controls, the continuity of the 
upper and lower value functions V^^(i, x) in t is quite subtle. To prove that, we need to establish a modified 
principle of optimality and fully use the coercivity conditions. It is interesting to indicate that the assumed 
coercivity conditions that ensuring the finiteness of the upper and lower value functions are actually sharp 
in some sense, which was illustrated by a one-dimensional LQ situation. 

For some other relevant works in the literature, we would like to mention [18, 14, 13, 1, 25, 22], and 
references cited therein. 

The rest of the paper is organized as follows. In Section 2, we make some brief observations on an AQ 
two-person differential game, for which we have a situation that the Isaacs' condition holds and the upper 
and lower Hamiltonians H {t,x,p) are quadratic in p but may be neither convex nor concave. Section 3 is 
devoted to a study of upper and lower Hamiltonians. The uniqueness of viscosity solutions to a class of HJ 
equations will be proved in Section 4. In Section 5, we will show that under certain conditions, the upper 
and lower value functions are well-defined and continuous. Finally, in Section 6, we show that the assumed 
coercivity conditions ensuring the upper and lower value functions to be well-defined are sharp in some sense. 



2 An AfRne-Quadratic Two-Person Differential Game 

To better understand two-person zero-sum differential games with unbounded controls, in this section, we 
look at a nontrivial special case which is a main motivation of this paper. Consider the following state 
equation: 

I y(s)=A(s,y(s))+Bi(s,y(s))wi(s) + B2(s,2/(s))M2(s), se[t,T], 

I y(i) == a;, 

for some suitable matrix valued functions A{- , •), Bi{- , •), and B2{- , •)• The state j/(-) takes values in R" 
and the control Ui{-) takes values in Ui — R™' (i ~ 1, 2). The performance functional is given by 

^^ 1 



J{t,x;ui{-),U2{-)) = I Q{s,y{s)) + -{Ri{s,y{s))ui{s),ui{s)) 



1 (2 2) 

+ {S{s,y{s))ui{s),U2{s)) --{R2{s,y{s))u2is),U2{s)) 



+ ( 9i{s, y{s)),ui{s) ) + ( 02{s, y{s)), U2{s) ) 



ds + GiyiT)), 



for some scalar functions Q{- , •) and G{-), some vector valued functions 0i{- , •) and 02(' ^ O: ^^^ some matrix 
valued functions i?i(- , •), R2{- ,■), and S{- , •). Note that the right hand side of the state equation is affine 
in the controls 'Ui(-) and U2{-), and the integrand in the performance functional is up to quadratic in mi(-) 
and U2{-). Therefore, we refer to such a problem as an affine- quadratic (AQ, for short) two-person zero-sum 
differential game. We also note that due to the presence of the term ( S{s, y{s))ui{s), U2{s) ), controls wi(-) 



and U2(') cannot be completely separated. Let us now introduce the following basic hypotheses concerning 
the above AQ two-person zero-sum differential game. 

(AQl) The maps 

A: [0,T] xIR"^lR", Bi : [0,T] xIR"^lR"^™\ B2 : [0, T] x R" ^ 1R"^™% 

are continuous. 

(AQ2) The maps 

Q : [0, T] X R" ^ R, G : R" ^ R, i?i : [0,T] x R" ^ 5"S i?2 : [0, T] x R" ^ 5"% 
S': [0,r] xR"^R"^^"S 6I1 : [0, T] X R" ^ R"S 6I2 : [0, T] x R" ^ R"^ 

are continuous (where 5™ stands for the set of all (m x m) symmetric matrices), and Ri{t,x) and R2{t,x) 
are positive definite for all {t,x) € [0,T] x R". 

With the above hypotheses, we let 

Wl{t,x,p,ui,U2) = {p,A{t,x) + Bi{t,x)ui + B2{t,x)u2) +Q{t,x) + - {Ri{t,x)ui,ui) 

^ 2 (2.3) 

+ ( S{t, x)ui,U2 > - 2 ^ ^2(^7 X)U2,U2 ) + {9i{t,x),Ui) + { 02{t, x), M2 ) • 

Our result concerning the above-defined function is the following proposition. 

/i?i (t x) S(t x)'^ \ 
Proposition 2.1. Let (AQ1)-(AQ2) hold. Then the matrix ^, \' ^ ' , is invertible, and 

V S(t,x) -~R2[t,x) J 



M{t,X,p,Ui,U2) = - {Rl{t,x){ui ~Ui),Ui -Ui) + {S{t,x){ui -Ui),U2 -U2) 
-- ( i?2(i, X){U2 - U2), U2-U2) +Qo{t, X,p), 

where 

'ui\ ( Ri{t,x) S{t,xY \^'^ f Bi{t,xYp + 6i{t,x) 



U2J \,S{t,x) -R2{t,x)J \B2it,x)^p + 92{t,x)J' 

and 

Qo{t,x,p) ^ Q{t,x) + {p,A{t,x)) 

_\fBi{t,xYp + ei{t,x)VfRi{t,x) S{t,xY y^fBiit,xfp + ei{t,x) 
2\B2{t,x)'^p + 02{t,x)) \ S{t,x) -R2{t,x)) \B2(t,x)'^p + 02{t,x) 



(2.4) 



(2.5) 



(2.6) 



Further, (1*1,^2) given by (2.5) is the unique saddle point of (?ii,M2) '^ H(i,x,p, Ui, M2), namely, 

Wi{t,X,p,Ui,U2) < il{t,X,p,Ui,U2) < M{t,X,p,Ui,U2), V(ui,U2) £ Ui X U2, (2.7) 

and consequently, the Isaacs' condition is satisfied: 

H'^{t,x,p)= inf sup Ii{t,x,p,ui,U2) — sup inf H(t, x,p, mi, -^2) 

"iet/i«2G(72 ixset/s^iet/i (2.8) 

= H-{t,x,p) :^Qo{t,x,p), V(i,x,p) e [0,r] xR" x R". 

Proof. For simplicity of notation, let us suppress (i, x) below. We may write 

]H(p,Mi,M2) = - (i?l(ui -Ml), Ml -Ul) + {S{ui -Ui),U2 - M2 ) - - (i?2(u2 - U2),U2 - U2) +Qo, 



with ui, U2, and Qo undetermined. Then 

{p,A)+Q+{ B'[p + 0i,ui) + { Bjp + 6I2, M2 > + 2 ( ^i"i, ui) + {Sui,U2)--{ R2U2, M2 ) 
1 1 _ 

= H(p, Ui,U2) ^ -{ RlUi,Ui ) + ( Sui,U2 )-^{ R2U2, M2 ) - ( RlUi,Ui ) 

- {S'^U2,Ui) - {Sui,U2) + {R2U2,U2) +- (i?lMl,Ml) + {Sui,U2) -- (i?2U2,'"2) +Qo- 

Hence, we must have 

B^p + 6*1 = -RiUi - S''^U2, Sjp + 6*2 = -S'mi + i?2M2, 

1 1 (2.9) 

{p,A) +Q = - {RiUi,Ui) + {Sui,U2) -- {R2U2,U2)+Qo. 

Consequently, from the first two equations in (2.9), we have 

Ri S^ \fuA^_fBfp + 9, 

S -R2j\U2j \Bjp + 92 

Note that 



det ( ^' ^ ) =dct ( ^^ ,^_ , cz.-lcT^ ) = {-ir'dct{Ri)dctiR2 + SR^'S^)y^0. 



Ri S^\^ fRi 

S -i?2y ^0 -{R2 + SR^'S' 

R S'^ \ 
Thus, ( D I i^ invertible, which yields 

S —R2 J 



/Si\ fRi S^ - 


]-'( 


'Bfp + e^ 


\U2 ) \ S -R2, 


) [ 


^B'^p + 62 


Then from the last equality in (2.9), one has 






2\u2j \ S 


-i?2. 


J Km) 


= <^'^>+^-2(4' + ^2J 


[Ri 


-R2; 



Bfp + 01 
B^p + 02 



proving (3.5). Now, we see that 



lH(p, Ui,M2) = -- (i?2(M2 - U2),U2 - ^2 ) +Qo < Qo = H(p, ^1,^2) 
< 2 (^l("l -Ul),Ul -Ul)+Qo{t,X,p) = lH(p, Ml,-S2), 

which means that (■ui,'U2) is a saddle point of lH(i,a;,p,ui,U2). Then the Isaacs condition (2.8) follows easily. 
Finally, since i?i and i?2 are positive definite, the saddle point must be unique. D 

We see that in the current case, p i— > H^{t, x,p) is quadratic, and is neither convex nor concave in general. 
As a matter of fact, the Hessian H^ {t,x,p) of H^{t,x,p) is given by the following: 



pp 

T\T/ jy f_^ ^^ Qf^ ^^^T \ -1 / n. U ^\T 

HpJt,X,p) 



± i(B,{t,xYy(Ri{t,x) s{t,xY y'(B,{t,x) 

pp[hx,P) 2\B2{t,x)^J \S{t,x) -R2{t,x)J \B2{t,xy 



which is indefinite in general. 

We have seen from the above that in order the upper and lower Hamiltonians to be well-defined, the 
only crucial assumption that we made is the positive definiteness of the matrix- valued maps i?i(- ,•) and 
i?2(' , •)• Whereas, in order to study the AQ two-person zero-sum differential games, we need a little stronger 
hypotheses. For example, in order the state equation to be well-posed, we need the right hand side of the 
state equation is Lipschitz continuous in the state variable, for any given pair of controls, etc. We will look 
at the general situation a little later. 



3 Upper and Lower Hamiltonians 

In this section, wc will carefully look at the upper and lower Hamiltonians associated with general two- 
person zero-sum differential games with unbounded controls. First of all, we introduce the following standing 
assumptions. 

(HO) For 1 = 1,2, the set U^ C R"' is closed and 

OeC/„ i = l,2. (3.1) 

The time horizon T > is fixed. 

Note that both Ui and U2 could be unbounded and may even be equal to R™^ and R™^, respectively. 
Condition (3.1) is for convenience. We may make a translation of the control domains and make corresponding 
changes in the control systems and performance functional to achieve this. 

Inspired by the AQ two-person zero-sum differential games, let us now introduce the following assumptions 
for the involved functions / and g in the state equation (1.1) and the performance functional (1.2). We denote 

(HI) Map / : [0, T] x R" x t/i x t/2 ^ R" is continuous and there are constants cri, 0-2 > such that 

\f{t,x,Ui,U2)\ <L{{x)+\ui\"' +\U2D, V(t,x,Mi,W2)e [0, T] X R" X f/i X C/2 . (3.2) 

(H2) Map g : [0,T] x R" x C/i x C/2 ^ R is continuous and there exist constants L,c,pi,p2 > and 
/i > 1 such that 

c\uir-L{{x}^ + \u2r)<git,x,u^,U2)<L{{x)'' + \u,r)-c\u2r, ^^^^ 

V(t,x,Mi,U2) e [o,r] X R" X C/i X [/2- 

Further, we introduce the following compatibility condition which will be crucial below. 

(H3) The constants 'Ji,<J2,Pi,P2 in (H1)-(H2) satisfy the following: 

CT, <p,, i = l,2. (3.4) 

It is not hard to see that the above (II1)-(H3) includes the AQ two-person zero-sum differential game 
described in the previous section as a special case. Now, we let 

Wi{t,x,p,ui,U2) = (p, /(i,x,ui,U2)) +9{t,x,ui,U2), {t,x,Ui,U2) e [0,T] X R" X t/i X U2- (3-5) 

Then the upper and lower Hamiltonians are defined as follows: 



H (t,x,p) = inf sup Wl(t,x,p,ui,U2 
H^(t,x,p)^ sup inf lll(i,x,p, wi, M2) 



"'^^'"^^^^ (i,x,p)€ [0,r]xR"xR", (3.6) 



provided the involved infimum and supremum exist. Note that the upper and lower Hamiltonians are nothing 
to do with the function h(-) (appears as the terminal cost/payoff in (1.2)). The main result of this section 
is the following. 

Proposition 3.1. Under (H1)-(H3), the upper and lower Hamitonians H^{- , • , •) are well-defined and 
continuous. Moreover, there are constants C > 0, Xi^i^i > 0, (i = 1, 2, • • • , fc) such that 

PI I P2 

-L{x}''~L{x)\p\- C\p\^T^ < H^{t,x,p) <L{x)^^ + L{x)\p\+ C\p\^^^ , ^3^^^ 

y{t,x,p)e [0,r] xR" xR", 
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and 

k 

\H^{t,x,p)-H^{t,x,q)\<CY,{x)^^{\p\y\q\y'\p-ql 

y{t,x) e [0,T] xR", p,qe¥C' 

To prove the above, we will use the following lemma. 
Lemma 3.2. Let < a < p and c,N>0. Let 

0{r) ^ Nr" - crP , re[0,oo). 



Then 



ith 



Proof. From 



9{0) == 0, lim e{r) = -oo, 

'I 7-00 

we see that the maximum of 9(-) on [0, oo) is achieved at some point f G (0, oo). Set 

= e'{r) = Nar"-^ - cprP-^. 

Then 

cp 
which implies that the maximum is achieved at f given by (3.10), and 

max d(r) = max 9(r) = 9(f) = N\ ) '' — c( ) '' 

re[0,oo) relo.f] \ cp J \ cp J 

_!^ {p-a)a-^ _!^ 

I\ p-" ^ iV p-" 

cp-^ pf-" 



\cpJ \cp 



(3.8) 



max %)= me.x9{r)^9if) = {p-a)(—^)'-\ (3.9) 



-(f)*- ("») 



This proves our conclusion. □ 

Proof of Proposition 3.1. Let us look at H^{t,x,p) carefully {H^ {t,x,p) can be treated similarly). First, 
by our assumption, we have 

H(i,X,p, Ml,M2) < bl |/(t,X,Ul,U2)| +g{t,X,Ul,U2) 

< L{{x) +\uir + \u2r)\p\+L{{x)^ + \ui\p') - c\u2r (^-n) 

= L{(x)t^ + {x) \p\ + \p\ \uir + \uir)+L\p\ \u2r - (^^2^, 

and 

M{t,X,p,Ui,U2) > -\p\ |/(i,X,Mi,U2)| +g{t,X,Ui,U2) 

>-L{{x) +\uir + \u2r)\p\-L{{x)^ + \u2\P^)+c\ui\P^ (3.12) 

= -L{{x)^ + {x)\p\ + \p\\u2r + \u2r)-L\p\\uir+c\ui\PK 

Noting (Ti < pi, from (3.11), we sec that for any fixed {t,x,p,ui) E [0,T] x R" x R" x Ui, the map 
U2 I— > lH(i, a;,p, ui, U2) is coercive from above. Consequently, since U2 is closed, for any given {t,x,p,ui) G 



[0,T] X R" X R" X Ui, there exists a, U2 = U2{t,x,p,ui) G U2 such that 

H'^{t,X,p,Ui) = sup H(t,X,p, Mi,U2) = sup ]H(t,X,p, Mi,U2) = ]H(f,X,p, Mi,U2) 

"26^2 U2£U2,\U2\<\U2\ 



<L{{x)^ + {x) \p\ + \p\ \uiP + \uir)+L\p\ \u2p ~ c\u2r 
<L{{x)'^+{x)\p\ + \p\\u^r + \u,r) + {p2-a2) 



a^'{L\p\Y'^^ (^-^^^ 



P2-<^2 



< L((a;) ^ + {x) \p\ + \p\ \uiP + \ui\P') + K2\p\^^ , 
where 

'9 -^ \ P2 ^^2 



--<-->(^)^ 



P2 

Here, we have used Lemma 3.2. On the other hand, from (3.12), for any {t, x,p, ui) E [0, T] x R" x R" x Ui, 

we have 

H^{t,x,p,ui) ~ sup ]H(i,x,p, Mi,U2) > H(i,x,p, Mi,0) 

«2ec/2 (3.14) 

> -i((a;) ^ + (x) H) - L\p\ \uir+c\u,r- 
By Young's inequaUty, we have 

L\p\ \u,r < ^\u^r + KM^^. * - 1,2, 

for some absolute constants Ki (depending on L,c,pi,ai only), which leads to 

^\u,\P' < c\u^\P^ ~ L\p\\u^\''^ + K^\p\^^ , ^ = l,2. (3.15) 

Hence, combining the first inequality in (3.13) and (3.14), we obtain 

C - P2 

^\u2r < C\u2r - L\P\ \u2r + K2\p\^^^ 

<L{{x)^ + {x)\p\ + \p\\uir + \ui\P')-n+{t,x,ui) + K2\p\^^ (3.16) 

< 2L((x) ^ + {x)\p\ + \p\ lu^r) +{L- c)\uir + K2\p\^^ = K2{\xl |p|, |ui|). 

The above implies that for any compact set G C [0, T] x R" x R" x Ui, there exists a compact set U2{G) C U2, 
depending on G, such that 

H^{t,x,p,ui) = sup Ii{t,x,p,ui,U2), y{t,x,p,ui) e G. 

U2eU2(G) 

Hence, 'H^(- , • , • , •) is continuous. Next, from (3.14), noting cti < pi, we have that for any fixed {t,x,p) G 
[0,r] X R" X R", the map ui i-^ H^{t,x,p,ui) is coercive from below. Therefore, using the continuity of 
'H^(- , • , • , •), one can find a mi = ui{t, x,p) such that (note (3.15)) 

H^{t,x,p)= inf sup H(i, x,p, Ml, M2) = inf H^{t,x,p,ui) =H^{t,x,p,ui) 

uieUiu2eU2 uieUi 

> inf H(i,x,p,Mi,0) > inf { - LUx) >" + {x) \p\) - L\p\\ui\'^ + c\ui\p^\ (3-17) 

>-L((x)^ + (x)H)-i?iHH^. 

This means that H^{t,x,p) is well-defined for all {t,x,p) E [0,T] x R" x R", and it is locally bounded from 
below. Also, from (3.13), we obtain 

H^{t,x,p)^ inf sup WL{t,x,p,ui,U2) < sup M{t,x,p,0,U2) ^ 7i^{t,x,p,0) 

uieUiu2eU2 u2eU2 (3.18) 

< L{{x) ^' + {x) \p\) + K2\p\^^ . 
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This proves (3.7) for if+(- , • , •)• 

Next, we want to get the local Lipschitz continuity of the map p h- > H^{t, x,p). To this end, we first let 

{f — PI P2 ') 

uieUi\ -\ui\P' < 2L{{x) '' + (x) \p\) + Ki\p\7T^ + K2\p\^^^ + Ij, Va;,pelR", 

which, for any given x,p G R", is a compact set. Clearly, for any ui Cz Ui\ Ui{\x\, |p|), one has (note (3.15)) 



c\ui\P' - L\p\\uir > ^\uir - Ki\p\7T^ > 2L{{x) >" + {x) \p\) + K2\p\^^^ + 1. 

Thus, for such a ui, by (3.14) and (3.18), 

n+{t,x,p,ui) > -L{{x)^ + {x) \p\)-L\p\ l^ir^ +c|ui|''i 

> LUx)^' + {x)\p\)+K2\p\'^'^ ^l>H+{t,x,p) + l= inf 'H+{t,x,p,ui) + 1. 



Hence, 



inf H^(t,x,p,ui) ^ inf H^{t,x,p,ui). 

uiEUi ui€Ui{\x\,\p\) 



Now, for any m G C/i(|a;|, \p\), by (3.16), we have 



-,\u2r<K2{H\p\,\u^\)<K2{\x\,\p\), 



(3.19) 



(3.20) 



(3.21) 



for some iir2(|a;|, |p|). Hence, if we let 

U2{\x\,\p\) = {^2eU2\^\u2r<K2{\x\,\p\)}, 

which is a compact set (for any given x,p E R"), then for any {t,x,p) E [0,T] x R" x R" 



H^{t,x,p)= inf sup ti{x,p,ui,U2) 

MieC/i(|a;|,|p|)„2g^2(|j.|jp|) 



(3.22) 



This implies that H^{- ,• ,•) is continuous. Next, we look at some estimates. By definition, for any ui € 
t/i(|a:;|, \p\), we have 

{ PI P2 'I 

(X) '' + (X) \p\ + |p|Pl— 1 + \p\P2-'2 K. 

Therefore, 

zi. 

\uir < C{ (x) ^ + (x) IpI + \p\^^ + \p\T^Y' 

{ P"1M (^1 ^\ ^1 '^1P2 ^ 

(x) "1 + (x) fl IpI PI + IpIpI-"! + |p|''l(P2-"2) L 

Also, by (3.16), one has 

\U2\''^ < C{ (x) ^ + (x) bl + IpI Iwi^i + Iwil"! + |p|^i^} 

{r p"ip ^j__ ^\_ p"! P1P2 "I 

(x)^ + (x)|p|+ (x) PI + (x) PI |p|Pl + IpIpi-"! + |p|Pl<P2-P2) \p\ 

+ ( X ) ^ + ( X ) IpI + IpI ^^T^ + \p\ ^^^^ I 

{ PIM Pj Pj+Pl PI P2 P1P2 I -I ^ 

(x) ^ + (x) IpI + (x) "^ \p\ + ^X) P^\p\ PI + |p|Pl-Pl + |p|P2-P2 + |p|pi(P2-P2) ^. 

Hence, 



(3.23) 



■{<• 



|W2r^ < C\ {X)^ + (.t) \p\ + (X) ^1 IpI + {X) PI \p\ PI + IpI"!--! + \p\P2-'-2 -\- |p|pi(P2--2)^ K 

{ P2P P2 P2 PI P2P P2 P1P2 P2(Pl+Pl) 

( X ) P2 + ( X ) P2 IpI P2 + ( X ) P1P2 IpI P2 + ( X ) P1P2 \p\ P1P2 

H P2P1 P2 P1P2 I P2 1 

P2(Pl-Pl) -(- Ip P2^P2 -|- p Pl(P2-P2) P2 ^. 
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Consequently, for any (t,x) € [0,T] x R", p, g G R" and Ui G ?/i(|a;|, |p| V \q\) {i = 1,2), we have (without 
loss of generality, let \q\ < \p\) 

\Wl{t,x,p,Ui,U2) -Ii{t,x,q,Ui,U2)\ 

< \p - q\\f{t,x,u,,U2)\ < L{{x) +\u,\''' + \u2p)\p - q\ 

{ ^IM ^72^ g"! CTi CT2 <72 c^i 0-2 

(x) + (x) PI + (a;) ^2 + (a;) PI IpI PI + (x) P2 IpI P2 + |p|pi-"i + |p|p2-<'2 
(3.24) 
-|-|t)| pi(P2-<'2) ^ Ipl P2(pi-"i) -|-( X ) P1P2 |n|p2+(a;) pip2 Inl P1P2 ^ IpI pi(P2-p'2) P2 fin — fll 

12 

^Cj2{x)^^{\p\V\q\r\p-q\. 

i=l 

Due to the fact that the infimum and suprenium in (3.22) can be taken on compact sets, we can prove the 
continuity of (t, x) h^ H^{t, x,p). □ 

A similar result as above can be proved under some much weaker conditions. In fact, we can relax 
(H1)-(H2) to the following. 

(HI)* Map / : [0,r] x R" x Ui x U2 ^- R" is continuous and there are constants ai,a2 > and 
/io,/^i,A*2 G R such that 

\fit,x,m,U2)\ < Ldx)^" + {x)^^\u,r + (a;) ^i«2r^), (3 25) 

V(i,x,iii,U2) G [0,r] X R" X C/i X C/2- 

(H2)* Map g : [0,T] x R" x C/i x t/2 — ^ IR- is continuous and there exist constants L,c,pi,p2 > and 
/2oj M17 M2 G R such that 

c{x)^'\uir - L{{x)^<^ + {x)^-\u2\'-) < g{t,x,ui,U2) ^3 26) 

<i((a;)^" + (a;)^l|ul|''l)-c(a;)^^|u2|''^ V(i, x, Wi, U2) G [0,r] x R" x t/i x U2. 

The following result can be proved in the same way as Proposition 3.1. 

Proposition 3.1*. Under (H1)*-(H2)* and (H3), the upper and lower Hamitonians H^{- , • , •) are we7]- 
defined and continuous. Moreover, there are constants C > 0, i^i > 0, and Xi G R (i ~ l,2,---,fc) sucii 
that 

-L{x)''° ~L{x)''"\p\ -C{x) "'pJ-^r'lpl^^I^ < H^{t,x,p) .327^ 

< L{x)i'° +L{x)i'»\p\ +C{x) P2— 2 \p\7I^^^ y{t,x,p) G [0,r] X R" X R", 

and 

fe 
\H^{t,x,p)-H^(t,x,q)\<Cj2{x)^^{\p\y\q\y'\p-q\, V(i, x) G [0, T] X R", p, g G R". (3-28) 

We point out that different from Proposition 3.1, there are more terms in (3.28) than in (3.8), and the 
expressions of Xi and i^i are a little more complicated. In fact, instead of (3.24) we can prove the following: 
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(for notational simplicity, we let \q\ < \p\)) 

\Ii{t,x,p,ui,U2) - ll{t,x,q,ui,U2)\ 

<\p-q\\f{t,x,u,,U2)\<L{{x)^<^ + {x)^^\u,r + {x)^-\u2r)\p~q\ 



<C 



{<^ 



[x) '^ "1 + (x) '^ PI 



Ml- 



7i(mo-Mi) 171 



^l(A^l-Ml) 



"bP 



P2 



a; 



A'2H 



1 + (^a;, 

'72(Ao-A2) 
P2 



/J1 + - 



,a;; ' ' ■ fi 

i[p2(M2-Ai)-<72(A2-/^iJ] 



IpI"! 



PI (P2 — o'2) m Pl(P2 — o'2) 



X 



M2 + 



^2(/^0~P-2J , 



P2 |p| P2 + (x) 



/J-2 + 



'^2(/^l ~/^2J '^l'^2A^l 



P1P2 m P2 



bl" 



^^ I '^2(a'1-P2) I glg2(P0-A'l) 



/J2 + - 



P2 ' P1P2 \p\'"^ + (x) 

P2 P2(Pl-'^l) Ipl P2(P1 -cri) 



,, <^2(P1-M2) I '^1'^2(a'0-A'i) . . g2(°"l+Pl) 

1 f'2^ „„ -r „, _ 



-\P\- 



I '^2(/^l~M2) I ^ltT2[P2(/^2~Ml)-'J-2(A^2-P-l)] 0-10-2 , 0-2 

3; \ ^^ P2 PlP2(P2-^2) Ipl PI (P2-<^2) P2 



P2(Pl-'^l) 



^2(/^2P2~A^2'^2) ^2M2 



|j>| P2(Pl-'^l) -\- (x) P2(P2-t^2) 



<^2 'I 
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Cy^{x)>^^{\p\W\q\y^\p-q\. 



i=l 



Note that (3.24) is a special case of the above with: 

4 Uniqueness of Viscosity Solution 

Consider the following HJ inequalities: 



Vt{t,x)+H{t,x,V^{t,x)) >0, (i,x) G [0,T] x R", 

y(T,x) < /i(x), xeR", 



and 



(4.1) 



(4.2) 



(4.3) 



f Vt{t,x)+H{t,x,V,{t,x))<Q, (i,x)€ [0,T]xR", 

I V'(T,a;) > /i(a;), xS R", 

as well as the following HJ equation: 

f Vt{t,x)+H{t,x,V^{t,x))=Q, (i,x)e [0,T]xR", 

\ V(T,.t) = /i(a;), xeR". 

We recall the following definition. 

Definition 4.1. (i) A continuous function V{- , •) is called a viscosity sub-solution of (4.1) if 

V(T,x)<h{x), VxeR", 

and for any continuous differentiable function (p{- , •), if (to, xq) G [0, T) x R" is a local maximum of (t, x) n- 

V{t,x) — ip{t,x), then 

Vtito,xo) + H{to,xo,(px{to,xo)) > 0. 

(ii) A continuous function V{- , •) is called a viscosity super- solution of (4.2) if 

V(T,x)>h{x), VxeR", 
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and for any continuous difFerentiable function (p{- , •), if (io, a^o) G [0, T) x R" is a local minimum of (i, x) n> 

V{t,x) — (p{t,x), then 

Vt{ta, xo) + H{to, xn,(px{t(3, xo)) < 0. 

(iii) A continuous function V{- , •) is called a viscosity solution of (4.3) if it is a viscosity sub-solution of 
(4.1) and a viscosity super-solution of (4.2). 

The following lemma is taken from [6]. 

Lemma 4.2. Suppose H : [0,T] x R" x R" -^ R is continuous. Let V{- , •) and V{- , •) be a viscosity 
sub- and super-solutions of (4.1) and (4.2), respectively. Then 

W{t, X, y) = V{t, x) - V{t, y), {t, x, y) £ [0, T] x R" x R" 

is a viscosity sub-solution of the following: 

I Wt{t, X, v) + H{t, X, Wx{t, X, y)) - H{t, y, -VK,(i, x, y)) > 0, (i, x, y) G [0, T] x R" x R", 

\ VK(T, X, y) < 0, (x, y) e R" X R". 



Now for HJ equation (4.3), we assume the following. 

(HJ) The maps H : [0,T] x R" x R" -^ R and h : R" -^ R are continuous and there are constants 
Kq > 0, /i > 1, and \i,i^i > (« = 1, 2, • • • , k) with 

Aj + (^ - l)j^j < 1, 1 < i < fc, (4.4) 

and a continuous function uj : [0, oo)'^ — > [0, oo) with property cj(r, s, 0) = 0, such that 

\H{t,x,p) - H{t,y,p)\ < Lu{\x\ + |y|, \p\, |x- y|), Vt G [0,T], x,y,p G R", 

fc (4.5) 

|i?(i,x,p)-if(i,x,(7)|<iro^(x)^'(|p|V|(z|)"'|p-g|, Vie[0,T], x,p,geR". 

and 

\h{x)-h{y)\<Ko{{x)\/{y))''-^\x-y\, Vx,yeR". (4.6) 

Our main result of this section is the following. 

Theorem 4.3. Let (HJ) hold. Suppose V{- , •) and V{- , •) are the viscosity sub- and super-solution of 
(4.1) and (4.2), respectively. Moreover, let 

\V{t,x)^V{t,y)l\V{t,x)^V{t,y)\<K{{x)\/{y)f-'\x-yl Vte[0,T], x,yeR", (4.7) 

for some f^ > 0. Tiien 

V{t,x) < V{t,x), V(i,x) e [0,r] X R". (4.8) 

A similar result as above was proved in [7], with most technical details omitted. Our conditions are a 
little different from those assumed in [7]. For readers' convenience, we provide a detailed proof here. 

Proof. Suppose (t, x) G [0, T) x R" such that 

V{i,x)~V{t,x) >0. 
Let Co,/3 > be undetermined. Define 

Q = Q{Co,(3) = {(t,x) e [0,r] X R" I (x) < (x) e^«(*-*^+^}, 
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and 

G= G(Co,/3) - [it,x,y) e [0,r] x R" x R" | (i,x),(t,y) e q}. 

Now, for (5 > small, define 



V'(t,x)=V'^'"^(i,x) 



„Co(t-t) 



' ^eni°ST^+c^«(*-*)J 



Then 



and 



V'(i,x) = f, V'(r,x) 



;s)' 



oCo(t-T) 



■>Pt{t,x 



ll^x{t,x) 



xip(t, x) 
<5(a;)2 



For any (t, x) G Q, we have 



Thus, Q is bounded and G is compact. We introduce 

^{t, X, y) = V{t, x) ~ V{t, y) ~ ^^1^ - a^(t, x) - ^^~^ 

e 1 — t 



{t,x,y) e G, 



where e > small and 



Clearly, 



< cr< 



V{t,x)~V{t,x) 



*(t, X, X) = y (t, X) - V{t, X) - O- - O- > 3(7 - 2(7 = cr > 0. 



(4.9) 



Since ^(- , • , •) is continuous on the compact set G, we may let (to, xq, yo) G G be a maximum of ^(- , • , •) 
over G. By the optimality of (to,xo,yo), we have 

V{to, Xq) - V{to, Xq) - aip{to,xo) -" = *(to, Xo, Xo) 

< ^{to, xo, yo) = V{to, xo) - 1/(to, yo) - ^i^i^ - aV'(io, xo) '"^^ " ^°^ 



T-t 



which implies 

Thus, 

Now, if to = T, then 

Hence, 



\xo -2/ol' 



<F(to,xo)-y(to,yo)<i^((xo)V(yo))'' |xo-yo| 



|xo-yo| < i^((xo) V(yo) 



,^-1 



(xo), (yo)<(x)e'^+^"(^-*^ 



*(r,xo,yo) = h{xo) - h{yo) " ^ " atp{T,xo) < Ko{{xo)V {yo))'' |xo-yo| 



e 

\2(p-l). 



<Xo^((xo)V(yo))^^" ^e<i^oi^(x)2(^-i)e2(^-i)[^+^"(^-*^l£. 
Thus, for £ > small enough, the following holds: 

*(T,xo,yo) <a< *(t,x,x) < *(io,xo,yo), 
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which means that to € [0,7). Next, we note that for (t,x) e (9Q) n [(0,r) x R"] , one has 

log 7^ + Co{t- t)^p, and < t < T, 
(x) 

which imphes 

tP{t, x) = e^ ^ oo, (5^0, uniformly in (t, x) e (dQ) n [(0, T) x R"] . (4.10) 

This imphes that for 6 > smah (only depending on /3), 

{to,xo,ya) e GU [{0} x R" x R" 
By Lemma 4.2, we have 

n^ I u \ ^ , lT(^^ 2(xo - yo) , A / 2(yo - xq) 
0< CT-^HtcaJo) - ^7-7T + ^(^^o,a;o, V aipx{to,xo)'j - H[to,ya, 

I u \ '^ , uf+ 2(xo-yo) , I f, A rrf, 2(xo-2/o)\ 
= crVt(io,a;o) - ;^7— 7 + ^(^*0, a;o, \- aip^(to,xo)j - H\^to,xo, j 

.rrf. 2(xo-yo)\ / 2(.xo-2/o) 

+ii to,a;o, - H[to,yo 



e / \ £ 

k 



<cr-0t(io,a;o)-;^r7T+^o^ (xo) -^'(^ — + (T\ipx{to,xo)\j cr |V'a;(io,a;o)| 

i—l 

, / 2|xo-yo| I |\ 

+cj(^|xo| + \yo\, , \xo - ya\j 

k 

--^(to,xo)-^-^- + .ifo5:(xo) '(2K((xo)V(yo)) +^^^^j J^^^^ 

, / 2|xo-yo| 

+oj[\xo\ + \yo\, 



< 



-,Fo -2/0 

Note that (io, a^o, J/o) = (^Ce: 2;o,e, 2/o.e) G C!{Co,f3) (a fixed compact set). Let e — ?• along a suitable sequence, 
we have |xo,e — i/o,el ~^ 0. For notational simplicity, we denote (to.e:2;o.e, j/o.s) ^" ita,XQ,XQ). In the above, 
by canceling a, and then send e — )■ and ct — >■ 0, one obtains (canceling a) 

^<- ^°^(^"'"°) +i^o^(xo)^-(2J^(xo)^-^^"^^^°'"°^ 



T-t- <5 "^^ ./ V ^ '^' y J(a;o) 

i—l 

<-{Co-i^oE(2i^)-}%^--(Co-Xo)^^. 

i—l 

Thus, by taking Cq > Kq, we obtain a contradiction, proving our conclusion. □ 

We now make some comments on the uniqueness/non-uniqueness of viscosity solutions. First of all, let 
us look at the following example which is adopted from [5, 4], 

Example 4.4. It is known that there are two different bounded strictly increasing continuous diffcrcn- 
tiable functions /i : R — > R (z = 1, 2) such that 

b{x) EE m^\x)) = m^\x)), X e R. 

Further, if we define 

X\t;xo)^Mt + fr\xo)), teU 
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then X^{- ; xq) and X'^{- ; xq) are two different solutions to the foUowing initial value problem: 

-X{t;xo) = b{X{t;xo)), t e U, 
X(0;xo) = xq. 

By defining 

V%t, x) = h{X'{T - t; x)), (i, x) e [0, T] x R, 

we obtain two different viscosity solutions to the following HJ equation: 

i Vt{t,x)+b{x)V,it,x)^0, {t,x)e[0,T]xn, ^^^^^ 

Therefore, the viscosity solution to the above HJ equation is not unique in the set of continuous functions. 
However, we note that in the current case, 

H{t, X, p) = b{x)p, {t, X, p) G [0, T] X R X R. 

Thus, 

\H{t,x,p) - H{t,x,q)\ <c\p ^ ql vte[o,r], x,p,qeiR, 

which means that (4.5) holds with fc = 1, Ai = J^i = 0. Hence, for any /i > 1, as long as (4.6) holds, viscosity 
solution to (4.11) is unique in the class of continuous functions satisfying (4.7). 

Example 4.5. Consider 

-x^ -axV,j,{x) + \V^{x)\^ ^Q, xeR, 



with a > 0. Thus, 

Then let 

We should have 

Hence, 



H{x,p) = —x'^ — axp + p^, (a;,p) e R 

V{x) = Ax^ a; G R. 

= -l-2aA + 4A2. 

2a ± V4a2 + 16 a ± \/^?T4 
A = — 



8 
Therefore, there are two solutions to the HJ equation: 



,,_i_, , a ± y/a"^ + 4 , ^ 

V^{x) = V^— a; : xSR. 



Both of these solutions are analytic. Note that 

\H{x,p)-H{x,q)\ < (a\x\ + 2{\p\V\q\)^\p-q\, x,p,qeR, 



and 



,±,, , ,.,,^ \a±Va^+M 



|y±(x)-T/±(y)|<J Y^((^)V(2/))|^-2/|. 

Thus, in our terminology, /i = 2, fc = 2 with 

Ai =0, i^i = 1, A2 =0, i^2 = 1- 
Consequently, 

x, + {^l-l)u, = l, i^i,2. 

This means that although (4.4) is satisfied, the corresponding HJ equation has more than one viscosity 
solution. This example shows that stationary problems are different from evolution problems, as far as the 
uniqueness of viscosity solution is concerned. 
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5 Upper and Lower Value Functions 

In this section, we are going to define the upper and lower value functions via the so-called Elliott-Kalton 
strategies. Some basic properties of upper and lower value functions will be established carefully. 

5.1 State trajectories and Elliott-Kalton strategies 

Let us introduce the following hypotheses which are strengthened versions of (HI) (H3). 
(HI)' Map / : [0,r] x R" x C/i x C/2 ^ H" satisfies (HI). Moreover, for some ^o,Aii,M2, 

\f{t, X, ui, M2) - f{t, y, ui,U2)\ 

<[{{x)W{y)r + {{x)W{y)nu,r + {{x)W{y)f'\u2r]\x~y\, (5.1) 

V(i,ui,M2)e [0,T] X C/i X U2, x,yelR", 



and 



{f{t,x,ui,U2)~ f{t,y,ui,U2),x-y) < L\x - y\^ , 

y{t,ui,U2)e [0,r] xUiX C/2, x,2/GlR" 



(5.2) 



We note that condition (5.1) implies the local Lipschitz continuity of the map x h- > J{t,x,ui,U2), with 
the Lipschtiz constant possibly depending on luil*^! and |u2|'^^. This is the case if we are considering AQ 
two-person zero-sum differential games (see Section 2). On the other hand, condition (5.2) will be used to 
establish the local Lipschitz continuity of the upper and lower value functions, with the Lipschitz constant 
being of polynomial order of (x) V ( y). It is important that the right hand side of (5.2) is independent of 
('Ui,'U2); Otherwise, the Lipschitz constant of the upper and lower value functions will be some exponential 
function of ( x ) V ( j/ ) , for which we do not know if the uniqueness of viscosity solution to the corresponding 
HJI equation holds. By the way, we point out that (5.2) does not imply the local Lipschitz continuity of the 
map X H- > /(i, X, ui, 7*2)- For example, /(x) — xa, with x e R. 



(H2)' Map g : [0,T] x R" x t/i x L/2 ^ R satisfies (H2). Moreover, 

|5(t,x,Mi,M2) - g{t,y,ui,u2)\ <[{{x)y {y)Y + |ui| ^ + \u2\ ^ ]|a;-y|, 

V(t,Mi,U2) e [0,T] X C/i X C/2, x,y eR". 

Also, map h : R" — > R is continuous and 

\h{x)-h{y)\<L{{x)y{y)Y-'\x^y\, Vx,yeR", 

\hm<L. 



(5.3) 



(5.4) 



Further, the compatibility hypothesis (H3) is now replaced by the following: 
(H3)' The constants ai,a2, pi, P2, 1^ appear in (H1)'-(H2)' satisfy the following: 

(7i^ < Pi, i = 1,2. (5.5) 

Let us first present the following Gronwall type inequality. 
Lemma 5.1. Let 9, a, (3 : [t, T] -^ R+ and 9o > satisfy 

0{sf <el+ I [a{r)e{rf + P{r)e{r)\ dr, s G [t, T]. (5.6) 
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Then 



0{.s)<e^ -17 "(^'"^^^o + le-fT "("'"^^ f I3{r)dr, s G [t, T] . (5.7) 

2 Jt 



Proof. First, by the usual Gronwall's inequality, we have 

0{s)^ < ef>^^'>''^9l + r e^>^^^''^ I3{r)e{r)dr 

Then 

^V0(^= ^0W-^0(^) = ie(.)-^e/>(^)'^^/3(.)0(.) < leI>^'^'^P{s). 

Consequently, 



2 



proving our conclusion. □ 

We now prove the following result concerning the state trajectories. 

Proposition 5.2. Let (111)' hold. Then, forany{t,x) e [0,r)xlR", (ui(-), M2(-)) eU^'[t,T]xU2^[t,T], 
state equation (1.1) admits a unique solution y(-) = y{- ;t,x,ui(-),U2{-)) ^ yt,x{'). Moreover, there exists a 
constant Co > only depends on L, T, t such that 

{yt,As))<Co{{x)+J\\u^{r)r+\u2{r)r)dr}, se[t,T], (5.8) 

\yt.As) -x\< Co{{x){s -t)+ f {\ui{r)r + \u2{r)r)drY s e [t,T]. (5.9) 

and for {t,x) e [0,r] x R" withte [t,T], and yi^-) = y{- ;t,x,uii-),U2i-)) 

\ytAs)-yU^)\<Co{\x-x\ + {x)ii-t) + J {\u,{r)r+\u2{r)r)dT}, s e [t,T]. (5-10) 

Proof. First, under (HI)', for any (t,x) e [0,T) x R", and any (ui(-), U2(-)) ^ U^'[t,T] x U2^[t,T], the 
map y i— )■ f{s,y,ui{s),U2{s)) is locally Lipschitz continuous. Thus, state equation (1.1) admits a unique 
local solution y{-) — y{- ■,t,x,ui{-),U2{-))- Next, by (5.2), we have 

{x,f{t,X,Ul,U2)) = {x,f{t,X,Ul,U2) - fit,0,Ul,U2)) + {x, f {t,0, Ul, U2) ) 

KLlxl^ + LlxKl + lml"' + \U2D, y{t,x,ui,U2) e [0,T] x R" x Ui x U2. 

Thus, 

{y{s)A^{xy + 2 j {y{r),f{r,y{r),u,{r),U2{r)))dr 

< (x)^ + 2 J'\(^{y{r))^ + (yir)) (1 + \u,{r)r + \u2{r)r))dr. 
Then, it follows from Lemma 5.1 that 

(y(s)) < e^(^-*) (x) +Le2^(^-*) /' (l + \ui{r)p + \u2{r)\''')dr. 
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This implies that the solution y{-) of the state equation (1.1) globally exists on [t,T] and (5.8) holds. Also, 
we have 



\y{s)-x\'^ = 2 / {y{r) -xJ{r,y{r),ui{r),U2{r)))dr 

< 2 / (L\y{r) - x\'^ + {y{r) - x, f{r,x,Ui{r),U2{r))) ]dr 

<2L I {\y{r) - x]^ + \y{r) - x\{{x) +\ui{r)y' + \u2{r)y^))dr. 

Thus, by Lemma 5.2 again, we obtain (5.9). 

Now, for any {t,x),{t,x) e [0,r] x R", with < t < t < T, denote yt.x(.-) = y{- ;t,x,ui{-),U2{-)), and 
yi,xi-) = y{-;ti^,ui{-),U2i-)). Then for s G [t,T], we have 

\yt.x{s) - yLx{s)\'^ = \yt,x{t) - sp 

+2 / {yt,x{r) - yi.x{r)J{r,yt,x{r),ui{r),U2ir)) -f{r,yi^s,{r),ui{r),U2{r)))dr 
< \yt.x{t) - A^ + 2L f \yt.x{r) - yi,x{'^)\^dr. 
Thus, it follows from the Gronwall's inequality that 

\ytAs) - yiM-^)\ < e^^^'''|yt,.(i) - ^1 < e^(^-*Hl^ - ^1 + IvtAt) - A) 

<e^'-'-''^Ux-x\+Le^^'-^-'U{x){i-t)+ j \ui{r)y^ dr + j \u2{r)\''^ dr\\ 
<c[\x-x\ + {x){i-t)+ f (|ui(r)|"i + |u2(r)r^)rfr}. 

This completes the proof. □ 

From the above proposition, together with (H2)', we see that for any Ui{-) G U^'[t,T] (which is smaller 
than Up [t, T]), i = 1,2, the performance functional J{t, x; mi(-)) ■W2(-)) is well-defined. Let us now introduce 
the following definition which is a modification of the notion introduced in [11]. 

Definition 5.3. A map a\ : U\^,T\ -^ Z^f°[i,r] is called an Elliott-Kalton (E-K, for short) strategy for 
Player 1 if it is non- anticipating, namely, for any 'U2(-), U2(') G ^2 [^: ^]i ^^'^ '^^Y ^ ^ [^i ^]) 

ai[w2(-)]('S) = Q:i["2(-)](s)i a.e. s G [t,f], 

provided 

Mi(s) = u\{s), a.e. s G \t,v\. 

The set of all E-K strategies for Player 1 is denoted by Ax^,T\. An E-K strategy a2 : U\^,T\ -> U'^\t,T\ 
for Player 2 can be defined similarly. The set of all E-K strategies for Player 2 is denoted by A2\t,T\. 

Note that as far as the state equation is concerned, one could define an E-K strategy a\ for Player I as 
a map a\ : U2^[t,T] — >■ U'[^[t,T]. Whereas, as far as the performance functional is concerned, one might 
have to restrictively define ai : U2^[t,T] — > Wf^[t,T]. We note that the numbers (7i,(72, Pi,P2 appeared in 
(H1)'-(H2)' might not be the "optimal" ones, in some sense (for example, cti and a2 rnight be larger than 
necessary, and pi and p2 could be smaller than they should be, and so on). Our above definition is somehow 
"universal". The domain U2[t,T] of ai is large enough to cover possible U2{-) in some larger space than 
142^ [t,T], and the co-domain l4^[t,T] is large enough so that the integrability of ai[u2(')] is ensured and 
the supremum will remain the same due to the density oi U^[t,T] in Ui^[t,T]. In what follows, we simply 
denote 

U,[t,T]=Ur[t,T], 2 = 1,2. 
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Recall that G f/i (i = 1,2). For later convenience, we hereafter let Ui{-) G l4i[t,T] and u^i-) G l42[t,T] be 
defined by 

w?(s)-0, u%s)^0, VsG[t,T], 

and let a° G Ai [t, T] be the E-K strategy that 



a° 



°[^i2(-)](s)=0, VsG[t,T], ^i2(.)GZ^2'[i,7^] 



We call such an a^ the zero E-K strategy for Player 1. Similarly, we define zero E-K strategy a!] G ^2 [^,2^] 
for Player 2. 

Now, we define 

V^{t,x)— sup inf J(i,x; ui(-),a2[ui(-)]), 

.,eMlt,T]M-)euAt,T] (i,x)G[0,T]x]R", (5.11) 

V" (t, a;) = inf sup J{t,x;ai[u2{-)],U2{-)). 

Qie^i[t,T]„2(.)£jy2[t,T] 

which are called upper and lower value functions of our two-person zero-sum differential game. 
5.2 Upper and lower value functions, and principle of optimality 

Wc now introduce the following notations: For r > 0, 

i-T 



U,[t,T-r] = {m, G W4f,r] I f \u,{s)\P^ds < r} 



* = 1,2, 



and 



Ai[t,T:r]^[ai ■.U^[t,T] ^ Ui[t,T;r] \ ai G A[t,r]}, 

A2[t,T:r] = {a2 : Ul[t,T] ^ U2[t,T;r] \ 0^2 G A[t,r]}. 

We point out that although the upper and lower value functions are formally defined in (5.11), there seems 
to be no guarantee that they are well-defined. The following result states that under suitable conditions, 
V^{- , •) are indeed well-defined. 

Theorem 5.4. Let (H1)'-(H3)' hold. Then the upper and lower value functions V^{- , •) are well-defined 
and there exists a constant C > such that 

|y±(t,x)| <C(x)^, (f,x) G [0,T] xR". (5.12) 

Moreover, 



y+(t,x) = sup inf J{t,x;ui{-),a2[ui{-)]), 

a2eA2lt^T;N{\x\)]uii-)eUi[t,T-N{\x\)] 

V^{t,x)^ inf sup J{t,x;ai[u2{-)],U2{-)), 

aieAi[t,T:N(\x\)]^,^(^.)fzU2lt,T;N(\x\)] 

where A^(|a;|) = C (x) ^, for some constant C > 0. 

Proof. First of all, for any (i,x) G [0,T] x R" and wi(-) G Ui[t,T], by Proposition 5.2, we have 

{y{s)) < Co{ {x)+ r K(r)rVr} < Co{{x) +\\u,{-)\\li,^^^^^}. 



(5.13) 
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> 



> 



Then 

J{t, x; ui(.), 0) - y 5(s, y(s), ^^1(5), 0)ds + h{y{T)) 

[c\u,{s)r-L{y{s))^]ds-L{y{T))^ 

>-C^(^)^-C^hi(-)ll?.':(t,T)+/ c\u,{s)rds. 
Since (note ^ > 1) 

ll"i(-)ll?.':(*,T) = {l^\niir)rdry < {T - tf-^ j^ \u,{r)r^dr, 
we obtain (taking into account cri/i < pi) 



J(i,x;7ii(.),0)>-C(x)^+ / c\ui{s)r ^ C\ul{s)\'''^^ 



ds 



Consequently, 



>-C(x>^ + -/ |ui (s) I"! (is>-C( a; )^. 
2 Jt 



y+(i,x) = sup inf J{t,x:ui{-),a2[ui{-)]) 

a2e^2[t,T]«i(-)e^'i[t,T] 

> inf J(t,x-ui(-),al\ui(-)])>-C(x)t'. 

ui{-)eUi[t,T\ 



Likewise, for any U2(') G ^2 [^7 2^], we have 



(5.14) 



J{t,x-0,U2{-)) ^ / g(,s,y(,s),0,^i2(5))d5 + ;i(2/(r))<C(a;)^ 



(5.15) 



Thus, 

y^(i,x)= sup inf J(t,x; Mi(-),Q:2[ui(-)]) 

a2e-42[0,T]«i(-)eWi[t,T] 

< sup J(t,x;uO(-),a2K(-)])<C(x)'^. 

a2e^2[t,T] 

Similar results also hold for the lower value function V^{- , •). Therefore, we obtain that V^{t^x) are well- 
defined for all (t,x) e [0,T] x R" and (5.12) holds. 

Next, for the constant C > appearing in (5.12), we set 

AC 
c 

Then for any mi(-) G Z^i[i, T] \ Z^i[i, T; 7V(|a;|)], from (5.14), we see that 



J(i,x;ui(-),a^[ui(-)])>-C(a;)^ + - / \ui{s)\P'ds > C {x) ^^ 



> V+{t,x) 



sup 



a2eyl2[t,T]«i(-)eWi[t,T] 



inf J{t,x;ui{-),a2[ui{-)]). 



Thus, 



F+(i,x) 



sup 



inf 



a2e^2[t,T]«i(-)GWi[t,T;Af(|a;|)] 
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J(t,x;ui(-),a2[ui(-)])- 



(5.16) 



Consequently, from (5.15), for any ui(-) G l{i[t,T; N{\x\)], we have 

-C{x)'' <V+{t,x) < sup J{t,x;ui{-),a2[ui{-)]) 

a2eA2lt,T] 

<C{x)'' + C f \ui{s)rds-^f \a2[u,i-)]{s)rds 



2jt 



<C{x)'' + 2C"{x)^^- / \a2[ui{-)]{s)\''^ds 



T 



This imphes that 



2jt 

ij |«2[«l(•)]Wr^c^s<C(x)^ V«i(-)eWi[t,T;7V(|x|)], (5.17) 



with C — 2C{C + 1) > being another absolute constant. Hence, if we replace the original N{r) by the 
following: 



4C 
c 



and let 



It is clear that 



A2[t,T;r] = {a2 G A2[t,T] | f \a2[ui{-)]{s)\P^ ds < Ni\x\)}, 

then the first relation in (5.13) holds. 

The second relation in (5.13) can be proved similarly. □ 

Next, we want to establish a modified Bellman's principle of optimality. To this end, for any (f , x) G 

[0,T) X R" and te (t,T], let 

U^[t,t;r]^{u^{-)eU^[t,T]\ f \u^{s)\p^ ds < r}, i = l,2, 

and 

Ai [t, f; r] = |ai : U^ [t, T] -^ Ui [t, t; r] \ ai e Ai [t, T] | , 

A2 [t, i; r] = { a2 : ^l [t,T]^ U2 [t, t; r] | aa G ^2 [t, T] } . 

\ Ai[t,T;r]<ZA^[t,t-r]<ZA,[t,T], 

Thus, from the proof of Theorem 5.4, we see that for a suitable choice of iV(-), say, N{r) = C(l + r^) for 
some large C > 0, the following holds: 

V+{t,x)^ sup inf J(t,a;;ui(-),a2[ui(-)]): 

V^{t,x)^ inf sup J{t,x;ai[u2i-)],U2{-)). 

aie^i[t,t;7V(|2:|)]„2(.)g;^2[t,t;iV(|2:|)] 

We now state the following modified Bellman's principle of optimality. 

Theorem 5.5. Let (H1)'-(H3)' hold. Let {t,x) e [0,T) x R" and t e {t,T]. Let N : [0,oo) -^ [0,oo) be 
a nondecreasing continuous function such that (5.18) holds. Then 

V+it,x)= sup ini { f gis,yis),ui{s),a2[ui{-)]{s))ds + V+{i,y{t))}, (5.19) 



a2eA2lt,t;N(\x\)] ui{-)eUi[t,t:N(\x\)] I Jt 
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and 

V-{t,x)= inf sup { f gis,yis),a,[u2i-)M,U2is))ds + V-{i,yit))}. (5.20) 

We note that if in (5.19) and (5.20), Ai[t,t]N{\x\)] and Ui[t,t;N{\x\)] are replaced by Ai[t,T] and 
lAi[t,T], respectively, the result is standard and the proof is routine. However, in the above case, some 
careful modification is necessary. For readers' convenience, we provide a proof in the appendix. 

We point out that our modified principle of optimality will play an essential role in the next subsection. 

5.3 Continuity of upper and lower value functions 

In this subsection, we are going to establish the continuity of the upper and lower value functions. Let us 
state the main results now. 

Theorem 5.6. Let (HI)' (H3)' hold. Then V^{- ,•) are continuous. Moreover, there exists a constant 
C > and a nondecreasing continuous function N : [0, oo) -^ [0, oo) such that the following estimates hold: 

|t/±(t,x)-F±(i,x)|<C((x)V(x))^"'|x-x|, te [0,T], x,xelR", (5-21) 

and 

\V^{t,x)-V^{i,x)\<N{\x\)\t^t\^^^yf^''^^^^, Vi,tG [0,T], xeR". (5.22) 



Proof. We will only prove the conclusions for V^ {■,■). The conclusions for V (• , •) can be proved 
similarly. 

First, let < t < T, X, S e R", and let N{r) = C ( r ) ^ for some C > 0, such that (5.13) holds. Take 

ui{-)el(^'[t,T;Ni\x\V\x\)], aae^f [t,r;iV(|x|V|x|)]. (5.23) 

Denote U2{-) — a2[ui(-)]. Then 

/■^ / /"^ \- ^ 

/ lu.ir)]"^ dr < C U \u,{r)\P^dry' <C{{x)y{x)) "^ <C(x)V(x), i = l,2. 

Making use of Proposition 5.1, we have 

\ytA-'^)l\yt.A-^)\<Co[{x) V(x)+y {\u,{r)r+\u2{r)r)dr] < C( (x) V ( x ) ), s e [t,T], 

and 

\yt,x{s) -yt,x{s)\ < Co|x-x|, s e [t,T]. 

Consequently, 

\J{t,x;ui(-),U2(-)) - J{t,x;ui{-),U2{-))\ 

< / \9{s,yt,x{s),ui{s),U2{s)) - g{s,yt,xis),u{s))\ds+ \h{yt^x(T)) - h{yt.x{T))\ 



t 

T 



i(((yt,r.(s) ) V (yt,s(s) )Y + \m{s)\-^ + \u2{s)\-^'j \yt,x{s) ' ytd 

+L(^{ytAT))y{yt.x{T))y'\yt.x{T)-ytAT)\ 

<c{((x)V(x))^"' + (y \ui{s)rds)^ + [J \u2{s)rd.s)^]\x~x\ 
<C{{x)W(x)f^\x-x\. 
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Since the above estimate is uniform in (ui(-)i 012) satisfying (5.23), we obtain (5.21) for V^{- , •). 

We now prove the continuity in t. From the modified principle of optimahty, we see that for any e > 0, 
there exists an al € ^2[^,i; -^(|a;|)] such that 

V+{t,x)-e< inf { / g{s,y{s),u^{-),a'^[u,{-)]{s))ds + V+{t,y{i))} 

< / gis,yis),0,al[4i-)]{s))ds + V+ily{i)) 



<J L[{y{.s))^^c\a2[uU-)]i-s))r)ds + V+ii,x) + \V+ii,y{F})-V+{i,x)\ 
< I L{y{s))^ds + V+(t,x) + \V+(t,y{t))-V+{t,x)\. 



By Proposition 5.2, we have (denote U2(') — Q^li^il')]) 



\y(J)~x\<C {x){t^t)+ \ul{s)rds 



<C 



{x)(i-t) + ( / \uus)rds)''(i-t)— 



<c 



{x){t - t) + N(\x\){t - t)^^ 



Also, 



\y{s)\<Co[{x) + j \ul{s)rds\<N{\x\), se[t,t\. 

Hence, by the proved (5.21), we obtain 

\V+{i, y{t)) - V+{lx)\ < N{\x\ V \ymyit) - ^l < N{\x\){i- t)"^ 
Consequently, 



which yields 

On the other hand, 



V+{t,x)-V+{t,x) <N{\x\){t-t)^^ +e, 
V+{t,x)-V+{t,x) <N{\x\)(t-t)^^^^. 



g{s,y{s),ui{s),0)ds + V+{i,y{t))y 
Hence, for any e > 0, there exists a uf (•) G Ui[t,T;N{\x\)] such that 



V+(t,x)> inf 

u,i)eU,[t,T:N(\x\)] L Jt 



V+it,x)+e> J gis,yis),ulis),0)ds + V+it,yit)) 

>- [ L{y{s))''ds + c f \ul{s)rds + V+{i,x) ^ \V+{ly{t)) ~V+{i,x)\ 



>-J L{y{s))^^d.s + V+{t,x) - \V+{i,y{t)) - V+{t,x)\. 



Now, in the current case, we have 



\y{t)~x\<C {x){i-t)+ \u\{s)\'''ds 



<C 



{x){i-t) + [ I \ulis)rds)''ii~t)^r 
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<C 



{x){i-t) + Ni\x\){i^t)^r 



Also, 

\yis)\<Co[{x)+J \u\{s)rds\<Ni\x\), se[t,t\. 

Hence, by the proved (5.21), we obtain 

\V+{i, y{t)) - V+{i,x)\ < N{\x\ V |2/(t))|y(i) ~ x\ < N(\x\){i~ t)"^^ . 
Consequently, 



V+{t,x)-V+(t,x) >-N{\x\){t-t)~^^ -e. 



which yields 



V+{t,x)-V+{t,x) > -N{\x\){t-t)~^^. 
Hence, we obtain the estimate (5.22) for V^{- , •). 



5.4 Characterization of the upper and lower value functions 

Having the above preparations, we are now at the position to characterize the upper and the lower value 
functions of our differential game. Recall that in order Theorem 4.3 applies, we need the conditions (4.4)- 
(4.6) (for the maps H{- , • , •) and h{-) stated in (HJ) hold, and the upper and lower value functions have to 
be Lipschitz continuous in a particular form (see (4.7)). It is clear that the only thing that we need is the 
compatibility condition (4.4) for the numbers Xi^Vi appeared in (3.24) with the parameter /i appeared in 
(H2) and (H2)'. Let us now look at what we need here. From (3.24) (which is for the upper value function 
V^{- , •) only), and the similar set of conditions for lower value function T^^ (•,•), we should require: 



CTl^ 



0-2 M 



< 1, -^ <1 

Pi P2 

(a* - l)criP2 



<1, 



Pi - 0-1 
(/i- l)cr2Pi 



<1, 



(M - 1)^2 
P2 - 0-2 



< 1, 



Pl(P2 - 0-2) 

o■l<y2^J■ , (p - 1)0-1 



P2(pi-cri) 



<1, 



< 1, 



0'lCT2P , (p - l)cr2 



<1, 



P1P2 



Pi 



P1P2 



P2 



(5.24) 



o'iq'2 (^- i)o-2(q'i +pi) 
P1P2 P1P2 

(H - l)cria2 in - 1)0-2 



<1, 



Pl(P2-0-2) P2 

We now have the following proposition. 
Proposition 5.7. Let 

Then all the inequalities in (5.24) hold. 
Proof. First of all, we have that 

Pi 



< 1, 



0-10-2 JP- 1)0-1(0-2 +P2) 

P1P2 P1P2 

ip- 1)0-10-2 _^ ip - 1)0-1 

P2(pl 



<1, 



< 1. 



^1 



Pi 



fJ-f^i < Pi, 



1,2. 



(5.25) 



< 1 



(P - 1)'^» 
Pi - o-i 



< 1. 



Thus, under (5.25), the last two inequalities in the first line of (5.24) hold. Next, by the above equivalence 
and /i > 1, 

(/x- l)o-ip2 / (p- 1)P2 



< 



Pl(P2-0-2) p(p2-0-2) 



<1, 



and 



in - l)o-2/?i ^ (P- l)Pl ^ ^ 



P2(Pl 



'^1 



P(Pl -CTl) 
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Thus, the incquaUties in the second hne of (5.24) hold. Now, for the third hne, we have 

0-10-2M , (/i - l)o-i <• ^ I (^ - l)cri _ ^la2_ ^ 
P1P2 Pi ~ Pi Pi Pi ~ ' 

and 

0"i0'2M , (p - 1)0-2 < ^ I (p - l)g2 _ fJf2_ ^ , 

P1P2 P2 ~ P2 P2 P2 ~ 

This shows that the inequahties in the third hne of (5.24) hold. We now look at the fourth line. It is seen 
that 

(JlO'2 (p- l)o-2(o'l +Pl) O'iq'2 (p- 1)0-2 (gl +pcri) _ P^0-iCr2 

P1P2 P1P2 ~ P1P2 P1P2 P1P2 ~ 

and 

O'l0'2 (p - 1)0-1 (g2 +P2) ^ 0-1 0-2 (p- 1)0-1(0-2 +PO-2) _ P^0-iCr2 ^ , 
P1P2 P1P2 ~ P1P2 P1P2 P1P2 ~ 

Finally, for the fifth line, we have (making use of the inequalities in the second line of (5.24)) 
(p- 1)0-10-2 , (p- 1)0-2 0-2 r(p- l)a-iP2 



Pl(p2-0-2) P2 P2 Lpi(p2 - 0-2) 

and 

(p- 1)0-10-2 , (p- 1)0-1 _ CTi r(p- l)a-2pi 



+ p- 1 



/i — 1 



<^<1, 

P2 



<^<1. 



P2(Pl-0-i) pi Pi Lp2(Pl - Oi) J pi 

This completes the proof. D 

With the above result, we have the following theorem. 

Theorem 5.8. Let (H1)'-(H3)' hold. Then V^{- , •) are the unique viscosity solution to the upper and 
lower HJI equations (1.3), respectively. Further, if the Isaacs' condition holds: 

H+{t,x,p)^ H-{t,x,p), V(t,x,p) e [0,T] xR" xR", (5.26) 

then 

V+(t,x) = V-it,x), V(t,x) e [0,T] X R". (5.27) 

6 Remarks on the Existence of Viscosity Solutions to HJ Equa- 
tions. 

We have seen that under (H1)~(H3), the upper and lower Hamiltonians can be well-defined and the corre- 
sponding upper and lower HJI equations can be well-formulated. Moreover, we have proved the uniqueness 
of the viscosity solutions to the upper and lower HJI equations within a suitable class of locally Lipschitz 
continuous functions. On the other hand, we have introduced a little stronger hypotheses (H1)'-(H3)' to 
obtain the upper and lower value functions V^{- , •) being well-defined so that the corresponding upper and 
lower HJI equations have viscosity solutions. In another word, weaker conditions ensure the uniqueness of 
viscosity solutions to the upper and lower HJI equations, and stronger conditions seem to be needed for the 
existence. There are some general existence results of viscosity solutions for the first order HJ equations 
in the literature, see [17, 3, 23, 14, 7]. A natural question is whether the conditions that we assumed for 
the existence of viscosity solutions are sharp (or close to be necessary) . In this section, we present a simple 
situation which tells us that our conditions are sharp in some sense. 

We consider the following one-dimensional controlled linear system: 

( yis) = Ayis) + BiUiis) + B2U2is), s e [t,T], 

I yit) = 2;, 
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with the performance functional: 



J(t,x;ui(-),U2(-))- / [Qy{sf + Riui{sf - R2U2{sfYs + Gy{Tf 
where A,Bi,B2,A,Ri,R2,G E R. We assume that 

i?l,i?2 >0. 

Note that in the current case, 



(6.2) 



(6.3) 



Thus, 

fJ-o-i = Pi, 

which violates (5.5). In the current case, we have 



(Ti = (72 = 1, fl = pi = P2 = 2. 

i = l,2. 



H^{t,x,p) = H{t,x,p) = inf sup pf{t,x,ui,U2) + g{t,x,ui,U2) 



= Apx + Qx + inf 

Ml 



— inf 

"2 



i?2M2 ^ PB2U2 



RiUi + pBiUi 

Consequently, the upper and lower HJI equation have the same form: 

(Bl_Bl 

F(T,x) = Gx^ xeR. 



= Apx + Qx 



2^/B|__ B^^^2 

V4i?2 4i?i 



(6.4) 



2 j^2 

Vtit,x)+AxV^{t,x) + Qx^+[^- -^)T4(t,x)2 = 0, (t,x) G [0,T] x R, 

V4ii2 4iti / 



(6.5) 



If the above HJI equation has a viscosity solution, by the uniqueness, the solution has to be of the following 
form: 

V{t,x)^p{t)x^, (i, x) G [0, T] X R, (6.6) 

where p(-) is the solution to the following Riccati equation: 



p{t) + 2Ap{t) + g + (ll - i^)^^^)' = °' * ^ [°' ^]' 
p(r) = G. 

In another word, the solvability of (6.5) is equivalent to that of (6.7). 



(6.7) 



Our claim is that Riccati equation (6.7) is not always solvable for any T > 0. To state our result in a 
relatively neat way, let us rewrite equation (6.7) as follows: 



with 



= 2A 



{p + ap + jip^ +7 = 0, 
p{T) = .g, 

P-%-% 7 = 0, 9 = G. 



(6.8) 



Note that (3 could be positive, negative, or zero. We have the following result. 

Proposition 6.1. Riccati equation (6.8) admits a solution on [0, T] for any T > if and only if one of 
the following holds: 

o? - Ap-i > 0, 2f3g + a - V"^ - 4/^7 < 0; (6.9) 

The proof is elementary and straightforward. For reader's convenience, we provide a proof in the ap- 
pendix. 
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It is clear that there are a lot of cases for which the Riccati equation is not solvable. For example, 

a — /3 — "f ^ I, 

which violates (6.9). Also, the case 

a = 0, /3=-l, 7=1, g = -2, 

which also violates (6.9). For the above two cases, Riccati equation (6.8) does not have a global solution on 
[0, T] for some T > 0. Correspondingly we have some two-person zero-sum differential game with unbounded 
controls for which the coercivity condition (5.5) fails and the upper and lower value functions could not be 
defined on the whole time interval [0,r], or equivalently, the corresponding upper/lower HJI equation have 
no viscosity solutions on [0, T]. 

Acknowledgement. The authors would like to thank the referee for informing the authors several important 
references in the field, especially some most recent papers. Also, some comments made by Professor Y. Hu 
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Appendix 

Proof of Theorem 5.4- We only prove (5.19). The other can be proved similarly. Since Af(|a;|) and t are 
fixed, for notational simplicity, we denote below that 

Ui^Ui[t,i;N{\x\)], A2=A2[t,i;N{\x\)]. 

Denote the right hand side of (5.19) by V^{t,x). For any e > 0, there exists an al G A2 such that 

V+{t,x)-e< inf^ f / g{s,y{s),ui{s),al[ui{-)]{s))ds + V+{i,y{t))}. 
ui(-)eUi ^ Jt -• 

By the definition of V^{i,y{t)), there exists an al G ^2[i, T] such that 

V+{i,y{t))-e< inf J{t,y{t);ui{-),al[ui{-)]). 

ui(-)eUilt,T] 

Now, we define an extension S| e A2[t,T] of al e A2[t,T] as follows: For any 'Ui(-) G Ui[t,T], 

a2[ui[-)\{s) = S _ I 

a2['"i(-)|[fT]K'S), s e [t,T]. 
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Since al G -^2, we have 

* \a^[u,{-)]{s)rds - r \a-^[u,{-)]{s)rds < N{\x\). 
It Jt 

This means that a| G A2- Consequently, 

V+{t,x) > inf_ J(i,x;Mi(-),a|[ui(-)]) 
Mi(-)eWi 

= inf^ { / 5(s,y(s),'«i(s),a|[ui(-)](s))(is + J(i,2/(t);ui(-)|[j-yj,a5[Mi(-)|[t-T])} 

> inf^ I / 

> inf^ f / g{s,y{s),ui{s),al[ui{-)]{s))ds + V+{i,y{t))}-e>V+{t,x)-2e. 
ui(-)eUi ^ Jt -' 

Since £ > is arbitrary, we obtain 

V+{t,x) <V+{t,x). 

On the other hand, for any e > 0, there exists an q;| G A2 such that 



g{s,y{s),ui{s),al.[ui{-)]{s))ds+ ^ inf J(t, y(i); ui(-), a|[ui(-)) [ 

fii(-)eWi[t,T] J 



V^{t,x) - £ < inf_ J(t,a;;ui(-),a2[ui(-)])- 



Also, by definition of V^(t,x), 



V+it,x)> inf^ f / gis,yis),ui{s),a'^[u,{-)]{s))ds + V+{i,y{t))}. 



ui(-)e.Ui ^ Jt 
Thus, there exists a u\{-) G lAi such that 

V+{t,x)+e> J g{s,y{s),ulis),al[ul{-)]{s))ds + V+{i,yit)). 
Now, for any ui(-) G Ui[t, T], define a particular extension ui{-) G Ui[t, T] by the following: 

[ wi(s), sG[i,T]. 



Namely, we patch uf(-) to ui(-) on [t,t). Since 

\ui{s)rds^ j \u\{s)rds<N{\x\), 

we see that ui(-) G Ui. Next, we define a restriction a| G ^[f, T] of a| G .42, as follows: 

a^[Mi(-)] =a|[ui(-)]. 
For such an al; ^'^ have 

l^+(i,2;(t)) > ^ inf _^ J(t,^/(i),Hl(•),«^2[^l(•)])• 
Hence, there exists a uf (•) G Wi[f, T] such that 

V+{i,y{t))+e>J{i,y{t),ul{-),al[ul{-)]). 
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Then we further let 

[ ulis), se[i,T]. 

Again, wf (•) G Ui, and therefore, 

V+it,x)+e> J g{s,yis),ulis),al[uli-)]is))ds + V+ii,yii)) 

> I g{s,y{s),ul{s),al[ul{-)]{s))ds + J(t,y{t),ul{-),a'2[ul{-)]) - e 

= J(i,x;Sf(-),a|[uf(-)])-e> inj: J(t, x; mi(-), a|[ui(-)]) -£> V+{t,x)-2e. 

ui(-)eUilt,T] 

Since e > is arbitrary, we obtain 

V+it,x) >V+{t,x). 

This completes the proof. | 

Proof of Proposition 6.1. Recall that we are considering the following Riccati equation: 

{■p + ap + Pp^ +7 = 0, 
V{T) - 5, 

Case 1. /3 ~ 0. The Riccati equation reads 

{p + ap + "f = 0, 
P{T)=9. 

This is an initial value problem for a linear equation, which admits a unique global solution p{-) on [0, T]. 
Case 2. (3^0. Then Riccati equation reads 

O; \ 2 4^^ _ Q,2 - 



P{T) = g. 



0, 



Let 



, Vl^^- 4/371 
''- 21/31 -"• 



There are three subcases. 

Subscase 1. a^ — 4/37 — 0- The Riccati equation becomes 



. P{T) = g. 

Therefore, in the case 

2/3.9 + a = 0, 

we have that p{t) = — ^ is the (unique) global solution on [0,T]. Now, let 

2/3.9 + a^0. 
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Then we have 



which leads to 



1 



pW + # 9+Wi 



F^-« 



1 R(rr A 2p-p{2Pg + a){T-t) 

— - P{T -t) = — 



2/3 



2/3^ + a 



Thus, 



Pit) 



2/3g + a 



2/3 2^- /3{2/3g + a)(T-t)' 



which is weh-defined on [0, T] if and only if 

2~{2f3g + a){T-t)^Q, t e [0,T]. 
This is equivalent to the following: 



{2f3g + a)T < 2. 



The above is true for all T > if and only if 



2(3g + a< 0, 



Subcase 2. c? — A/S-y < 0. The Riccati equation is 



P + P 



^^Yp} +" 



Hence, 

which results in 

By the terminal condition. 

Consequently, 



dp 



{p+%? + ^' 



= -I3dt, 



■ tan 



rl / , , oi 



= -(it + C. 



C = /3T+-tan"i 



K V 2/3 



tan 



~M^^^2p)\ =^l^(T-t)+Un-^ 



-n{'^2J3 



Then 



^^^^ = §+"*'''' {"^^^ " ^^ + *''''"' (^^^) }• 



The above is well-defined for t G [0, T] if and only if 

.1 2/3.9 + a 



— < tan 

2 2k/3 



+ K/?r<-, 



which is true for all T > if and only if /3 = 0. 

Subcase 3. o? — 4f3j > 0. The Riccati equation becomes 

2 



P+2p} -" 



If 



P + P 



(2/3g + a- 2k/3)(2/35 + a + 2k/3) = 4(3^ [g + — - k] [g + — + k] = 



2/3 



2/3 
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(Al) 



then one of the following 



p{t) = -—±K, te[0,T], 



is the unique global solution to the Riccati equation. We now let 

(2/?5 + a- 2np){2f3g + a + 2«:/3) = 4/3^ (g + ^ - «) (g + ^ + «) ^ 0. 

Then 

-I3dt. 



dp 



Hence, 



which implies 



with 



Then 



Consequently, 



2k 



In 



Pit) + %^K 



p{t) + 5^ + K 



2/3 



-pt + c. 



Pit) 



2/3 



p(t) + ^ + K 
,5+^-K 



Ce-2«/5*, 



O — 6 t; — 6 



5+2^+'* 

p(0 + ^ - « 

p(t) + ^ + K 



2/3.9 + a + 2k/3 

2K/3(T-t) 2/3g + g - 2k^ 
2^g + a + 2nj3 ' 



„m + ^ _ . - ,2./3(T-«) 2A9 + a-2K/3 
^^' ^213 2/35 + a + 2k/3 



P(t) + ^ + « 



Thus, p{-) globally exists on [0,r] if and only if 



2/3(7 + a + 2k/3 
which is equivalent to 

V'(t) = e2'='^(^-*) (2/3g + a - 2k/3) - (2;35 + a + 2kP) ^Q, Vt G [0, T] . 

Since il)'{t) does not change sign on [0,T], the above is equivalent to the following: 

< tlj{0)i^{T) = \e^''f^'^ {2f3g + a- 2k/3) - (2;35 + a + 2k;3)J (-4k/3), 

which is equivalent to 

^2KpT^2(3g + a- 2k/3) - (2/3.9 + " + 2k/3)J /3 < 0. 

Note when (Al) holds, the above it true. In the case /3 > 0, the above reads 

e^'*'^^(2/3g + a- 2k/3) < 2/3g + a + 2k/3, 
which is true for all T > if and only if 

2/3g + a~ 2k/3 < 0. 
Finally, if /3 < 0, then 

< e^^^^{2(3g + a- 2«;/3) - (2/3.g + a + 2k/3) 

= e-2"l^l^(-2|/3|g + a + 2k|/3|) - {-2\f3\g + a- 2k|/3|) 
= e-2-l^l^ [ - {2\/3\g -a- 2k|/3|) + e'^\^\^ {2\p\g -a + 2k[ 
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(A2) 



which is true for ah T > if and only if 

< 2\l3\g -a + 2k\I3\ ^ -{2Pg + a- 2k|, 

Thus, 

2Pg + a-2K\P\ < 0. 

which has the same form as (A2). This completes the proof. 
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